The Importance of Metrics and Evaluation Processes in Knowledge Based Systems

The knowledge engineering (KE) and software communities have several evaluation methods, and techniques, unfortunately, we have not reach an agreement about what the proper object of evaluation should be. We try to unravel the mystery around evaluation methods, exposing several important models, synthesizing their most important features into an evaluation process that keeps the best of them all. The discussion surveys some of the most important features to judge the quality of products. Through all the paper we try to hint answers to what should be measure and how. And finally, we exhibit a unified evaluation process, and how our knowledge acquisition methodology (KAM), KAMET, is embracing it. Through all this work we review evaluation processes in the light of modern KAMs and knowledge management (KM) techniques. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the CSIT copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Institute for Contemporary Education JMSUICE. To copy otherwise, or to republish, requires a fee and/or special permission from the JMSUICE. Proceedings of the 2 International Workshop on Computer Science and Information Technologies CSIT’2000 Ufa, Russia, 2000